FXIa inhibition has been a promising strategy for treating thrombotic diseases. Up to date, many small-molecule FXIa inhibitors have been identified; however, most of them exhibit undesirable selectivity over the homologous plasma kallikrein (PKal). By employing structure-based drug design strategies, we identified many novel selective FXIa inhibitors that have extra interactions with the S2 subsite of FXIa. Among them, compound 35 displayed good inhibitory activity against FXIa and high selectivity over PKal and even several other serine proteases. Additionally, 35 showed significant anticoagulant activity toward the intrinsic pathway without affecting the extrinsic pathway. In vivo, 35 exhibited significant antithrombotic activity without increasing the bleeding risk and obvious toxicity in mice, demonstrating that it could be a promising candidate for further research. This study first demonstrates the importance of the S2 subsite of FXIa, paving the way to design highly selective FXIa inhibitors for clinical uses.